A Parametrizable Dataflow Implementation of Optical Flow

An increasing number of products feature complex functionality through the use of embedded computers. By collecting large amounts of data from multiple sensors and interpreting and responding to complex scenarios in realtime, such devices can become “smart.” For a control algorithm to make a decision, data from the sensors must be processed before a certain deadline. These constraints may be hardor soft-real-time, and a combination of SRT or HRT applications may be executed on one platform [2]. In streaming applications such as multimedia and signal processing, the dataflow model of computation (MoC) is a good fit because these applications are inherently data-driven [3]. A dataflow program consists of a graph of actors that communicate tokens through channels. An actor can only fire (execute) if all of its input tokens are available, and if there is enough space on its outgoing channels to produce all tokens. Some dataflow flavours such as mode-controlled dataflow (MCDF) and scenario-aware dataflow (SADF) can be used to deal with dynamic behaviour within one application [4, 5]. In this research we consider the processing of image data to obtain direction vectors of moving objects, on the basis of which control algorithms can make decisions. In particular, we use the optical flow algorithm which is commonly used to track features in a sequence of moving images. To execute this algorithm on a contemporary multi-core streaming platform with high performance and while meeting timing constraints, it is useful have a dataflow implementation. Our implementation is based on the algorithm described in [1]. After refactoring in the C language and removal of any dependencies on the external libraries, we ported the algorithm to our CompSOC platform [2]. To distribute the functionality over multiple dataflow actors we recognized three main stages: the loading of frames into local memory, the initial detection of features in a frame, and the optical flow that detects identical features in two consecutive frames. After splitting the algorithm in these three stages it was parallelized and parametrized. The implementation thus obtained allows to exploit the benefits of the aforementioned mode-control and scenario-aware dataflow MoC, as the parameters can be used to set different modes or use-cases. Parallelization is straightforward as frames are split up in blocks and the calculation on each block is completely independent of other blocks. Parameters that can be adjusted are the frame size, the number of cores that will process a fraction of a frame in parallel, and optical flow parameters, namely: threshold, maximum allowed error and window size. The resulting graph for an implementation with two cores is depicted in Figure 1. Actor a is added to determine or receive the parameters, which are stored in its self-edge. These are forwarded to actor c, f and b.